Skip to content

Wait out the iOS retry backoff when the system refuses a delayed task - #80

Merged
dmurphy5 merged 2 commits into
masterfrom
dan/instant-failure-backoff
Nov 13, 2026
Merged

dmurphy5 merged 2 commits into
masterfrom
dan/instant-failure-backoff

Conversation

@danseethaler

Copy link
Copy Markdown

What could go wrong

On iOS, a retry's backoff exists only as the begin date of a background task: the library creates the next attempt right away and lets nsurlsessiond hold it until the date. If the daemon refuses the app's connection, that task fails immediately with NSURLErrorDomain -1, before its begin date and without sending anything. The library counted that as a transient failure and created the next delayed task at once, which failed the same way.

That loop ran at 300–600 attempts a second, each writing the entry and opening a new XPC connection. The app sat at about 200% CPU and grew to gigabytes of memory. We hit it on two simulators running diana's CI dev builds, which had no entitlements, so the daemon logged "does not have a bundle ID, rejecting connection" for every session. That build is fixed in openspacelabs/diana#9825. The library should still not spin when the daemon says no.

What this PR does

  • A refused delayed task isn't an attempt. A delayed task that ends before its begin date, with no response and no bytes sent, never reached the daemon. It emits no event and keeps its ordinal and request id. The app waits out the rest of the delay itself, then issues the attempt again (advanceAttempt: false). All three conditions are needed, so a clock change during the wait can't turn a real failure into this. The error code isn't checked, because -1 also comes with real failures.
  • Chunked parts get the same treatment. A refused part task keeps its begin date and refills once the date passes. Unlike a simple request, the refilled part still takes a new ordinal, as every part task does.
  • A floor as a backstop, simple requests only. A retry never starts within baseMs of the previous attempt's start. One that ended before its own start gets no task for baseMs. The floor lives in memory only.
  • TaskCompletion carries bytesSent (from countOfBytesSent) so the completion path can tell.

Android doesn't have this flaw: it waits out the backoff itself before making the request.

Review notes

How it was tested

  • swift test in ios/: 263 tests, 0 failures, 5 new. They cover every task refused for 10 s of fake time (bounded attempts, no tight loop), one refused task keeping its ordinal and X-Request-Id, a task failing after its begin date still counting, a task that sent bytes or got a 503 early still counting, and a refused chunked part. With the fix disabled the new tests fail.
  • In diana on an iPhone 15 simulator, with this branch in node_modules and the app built with CODE_SIGNING_ALLOWED=NO, so the daemon refuses it:
    • A stuck upload went from about 5,500 attempts per 30 s to 1 attempt per backoff.
    • Two new uploads backed off with growing gaps (about 14 s, 32 s, 74 s).
    • CPU stayed under 2%.
    • With v11.1.0 the same build spun at 199% CPU.
  • No JS changed, so yarn lint:ci, typecheck and test weren't run.

🤖 Generated with Claude Code

danseethaler and others added 2 commits November 13, 2026 08:04
A simple upload's retry backoff lived only in the earliestBeginDate of its
background task. When nsurlsessiond refuses the app's connection (a
simulator build with no entitlements), that delayed task completes at once
with NSURLErrorDomain -1, before its begin date and having sent nothing.
The completion was classified transient and scheduled another delayed task,
which failed at once too. The result was a tight loop of hundreds of
attempts a second, each one saving the entry and opening a new XPC
connection, so memory grew without bound and the backoff was never waited.

A delayed task that ends before its begin date with no response and no
byte sent is now treated as never started: no attempt event, no new
ordinal, and the coordinator waits out the rest of the delay in process
before issuing the same attempt again. Chunked part tasks get the same
treatment, without a rejection. As a safety net, scheduleRetry keeps the
next attempt at least baseMs after the previous one's start, and holds back
the task for baseMs when the previous attempt ended before its own start.

TaskCompletion now carries countOfBytesSent. Four existing tests delivered
a response to a delayed task before its begin date; they now advance the
fake clock first, as the daemon would.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@danseethaler
danseethaler requested review from dmurphy5 and a balanced review from Copilot November 13, 2026 14:24
@danseethaler
danseethaler removed the request for review from Copilot November 13, 2026 14:24
@arnonate

Copy link
Copy Markdown

does @dmurphy5's work conflict with this at all or will this slide in as an addition?

@arnonate

Copy link
Copy Markdown

From claude:

Mostly it slides in as an addition. I trial-merged Dan’s branch (openspacelabs/react-native-background-upload#80) against master and each of Dylan’s ten open PRs. Seven merge cleanly and three conflict, all of them small:

Dylan’s PR | Result with #80 -- | -- #69 storage cause | Conflict in ios/QueueCoordinator+Outcomes.swift #72 settings never reject | Same conflict, because it’s stacked on #69 #79 entry save durability | Conflict in CHANGELOG.md only #65, #67, #70, #71, #74, #76, #78 | Clean

A clean merge doesn’t prove the result compiles. Dan adds a required bytesSent field to TaskCompletion, so any other PR that builds one needs the new argument. Of Dylan’s PRs, only openspacelabs/react-native-background-upload#70 touches that area, and it changes many of the same iOS files. Dan’s PR says openspacelabs/react-native-background-upload#70 and #65 pass the test suite when combined with his branch. I didn’t run the suite on any combination myself, so the other clean merges are untested beyond the text merge.

Merge order doesn’t matter much. If #80 goes first, Dylan resolves the one-line conflict when he rebases openspacelabs/react-native-background-upload#69.

Mostly it slides in as an addition. I trial-merged Dan’s branch ([#80](https://github.com//pull/80)) against master and each of Dylan’s ten open PRs. Seven merge cleanly and three conflict, all of them small:

Dylan’s PR Result with #80
openspacelabs/react-native-background-upload#69 storage cause Conflict in ios/QueueCoordinator+Outcomes.swift
openspacelabs/react-native-background-upload#72 settings never reject Same conflict, because it’s stacked on #69
openspacelabs/react-native-background-upload#79 entry save durability Conflict in CHANGELOG.md only
#65, #67, #70, #71, #74, #76, #78 Clean
#69 and #72: Dan adds one line, attemptStarts[id] = nil, at the end of the settle function. #69 rewrites the end of that function and drops the return true beside it. Whoever merges second keeps Dan’s line and #69’s version of the ending. Dan already calls this out in his review notes.
#79: both PRs add a changelog entry in the same spot. Keep both.

A clean merge doesn’t prove the result compiles. Dan adds a required bytesSent field to TaskCompletion, so any other PR that builds one needs the new argument. Of Dylan’s PRs, only openspacelabs/react-native-background-upload#70 touches that area, and it changes many of the same iOS files. Dan’s PR says openspacelabs/react-native-background-upload#70 and #65 pass the test suite when combined with his branch. I didn’t run the suite on any combination myself, so the other clean merges are untested beyond the text merge.

Merge order doesn’t matter much. If #80 goes first, Dylan resolves the one-line conflict when he rebases openspacelabs/react-native-background-upload#69.

@dmurphy5
dmurphy5 merged commit 15eb3b4 into master Nov 13, 2026
5 checks passed
dmurphy5 added a commit that referenced this pull request Nov 13, 2026
Brings in #65, #74, #78 and #80.

CHANGELOG.md: kept every entry; master's entries first, then this branch's
three (mutate() waits, onWarning, holdNext).
src/__typetests__/define.ts: kept both blocks (this branch's onWarning
checks and master's takeDiagnostics() checks).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
dmurphy5 added a commit that referenced this pull request Nov 13, 2026
Brings in the updated base, which now has #65, #74, #78 and #80 from
master.

CHANGELOG.md: kept every entry; master's entries first, then #71's, then
this branch's (row hooks, fake settle() timing).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
dmurphy5 added a commit that referenced this pull request Nov 13, 2026
Brings in #65, #74, #78 and #80.

CHANGELOG.md: kept every entry; master's entries first, then this branch's
two (setWifiOnly() no-op write, E_STORAGE cause).
ios/QueueCoordinator+Outcomes.swift: kept this branch's throwing settle()
and added #80's `attemptStarts[id] = nil` after `throttle.reset(id)`, where
master has it. Dropped master's `return true`, since settle() here returns
nothing. A user cancel whose journal write fails still throws before any
change, as master returns false before it.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
dmurphy5 added a commit that referenced this pull request Nov 13, 2026
Brings in the updated base, which now has #65, #74, #78 and #80 from
master.

CHANGELOG.md: kept every entry; master's entries first, then #69's, then
this branch's four.
ios/QueueCoordinator.swift: kept both in init. This branch's header
generation raise runs first, then master's recordBootSnapshot() call; the
recordBootSnapshot() function from master is kept as is.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
dmurphy5 added a commit that referenced this pull request Nov 13, 2026
Brings in #65, #74, #78 and #80.

CHANGELOG.md: kept every entry; master's entries first, then this branch's
four (sliced part builds, resume adopted tasks, expired at replace,
signposts).

#80's iOS changes auto-merged. Checked by reading: in
ChunkedCoordinator.partCompleted, a refused part keeps its begin date and
schedules refill(id), which now builds in slices and takes the remaining
wait from partBeginAt as before. In issue(), attemptStarts is set after
task.resume() and before throttle.reset(id).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
dmurphy5 added a commit that referenced this pull request Nov 13, 2026
Brings in #65, #74, #78 and #80.

CHANGELOG.md: kept every entry; master's entries first, then this branch's
(entry save survives a power loss).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
dmurphy5 added a commit that referenced this pull request Nov 13, 2026
Brings in #65, #74, #78 and #80.

CHANGELOG.md: kept every entry; master's entries first, then this branch's
two (late define() delivers held outcomes, configure({ definitions })).
src/__typetests__/define.ts: kept both blocks (this branch's
configure({ definitions }) checks and master's takeDiagnostics() checks).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants